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ABSTRACT 



A random method for generated binary trees is presented, 
and two forms of a class of one person games called "Tree 
Solitaire" which have such trees as their game trees are 
defined. After what "look-ahead strategy" means in terms of 
such games is discussed, a theorem on the most efficient use 
of unlimited look-aheads is proved, and a collection of strate- 
gies involving 0, 1, or 2 look-aheads per move is introduced- 

A method involving diagrams is presented for calculating 
the probability of winning under the various strategies over 
a restricted class of games* The superiority of one of the 
1 look-ahead strategies over the other is proved for games of 
the first form on this restricted class. For games of the 
second form in this class, all the introduced strategies have 
their chances of winning calculated, and these results are 
compared among themselves, with the result for the first form 
of the game, and with the results of Monte Carlo estimation of 
the chance of winning in a particular case. 

An approximate method for evaluating strategies from any 
given position is introduced, used to explain some of the pre- 
vious results, and suggest modifications of strategies already 
defined, which are then evaluated by Monte Carlo methods. 

Finally, variants on Tree Solitaire are suggested, their 
general implications are discussed, and using the methods 
already developed one of the most suggestive variants is 
studied and the results show a significant reversal frcm those 
of the original game, which is explained by the difference in 
the games on one particular. 
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INTRODUCTION 

In confronting the problem ol programming, computers 
to play Barnes such as ohess, where j^ood nlay seems to 
require the ability to "look ahead' 1 to future positions, one 
Is immediately -ivmrc that, desnite the noninutor 1 !! unced, the 
?*ajne trey i:; much too larce to allow looking ahead to the 
end of t"\e r.nme in unkliw the decision on where to move. 
Hrcoont fntk-rfs ^r.t Took to n etrtain iirtited depth in tno 
tr*:r: usin : fvfWit-Ona wnieh attempt to evaluate "how ;rood T 
the positions 86(01 ^re, nnd use what they find OS best they 
can. Certain techniques such as a-P ha* e been de/eioped 
to streamline this pro**£s, but no theoretic framework for 
the problem of look-ahead has been constructed. 

This paper reports an attempt to be^in such a 
construction by exa-nininp, a class of one person ^ames with 
simplified ^ejne tree, and evaluating various limited and 
unlimited look-ahead strategies for it. In doin/i so, a 
terminology for dealing with look-ahead is developed, and 
the kinds of distinctions and definitions that raay prove 
necessary or useful for further work are discussed. 

This is an exploratory study with no known predecessors 
in the literature* and so tiftnt organization and single-minded 
purpose have been avoided in the interest of presenting all 
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the varied results so for obtained from different approaches, 
and as many as possible of the conjectures and suggestions 
Tor further study thaz have occurred to the author. In 
addition much time has been spent on detailed explanations 
of problems and methods which, though not oarticularly deep, 
are nrobobly not particularly familiar either. 

Chapter 1 introduces the class o! games called 
Tree Solitaire (abbreviated TS) and its two forms (TSD and 
TmI}j b;/ explaining now the r.ame tree for such a p.ame is 
^cn^rated, and then turned into a f^amc. 

Chapter 2 discusses what is meant by T "loofc-a-hcad" 
in terms of this class of games, what is meant by "look- 
ahead strategy" , and how such are to be evaluated. It then 
proves a few simple results and defines the look-sJiead 
stratecies that will be dealt with In later chapters: STRAT 
and N03TRAT involving no look-ahead, HSTRAT and LSTRAT 
involving one, and BCTRAT, 2HSTRAT, and 2LSTRAT Involving 
two looK-aheads. 

In chapter 3 methods are Introduced which produce 
closed formulas for the probability of winning usin/s HSTRAT 
and LSTRAT in games of form TSD. It is proved that 
LSTRAT is always the better strategy in such Rames, 

Chapter *l uses methods similar to those of chapter 
'3 to evaluate, by means of computer program, all the look-ahead 
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strategics defined in chapter 2, over the class of names of 
form TSI. (Alternative evaluation by Monte Carlo methods 
is discussed for H3THAT and LSTRAT. J The strategies ore 
then compared as to their "effectiveness" in obtaining wins 
when various parameters involved in the generation of the 
r t smes are varied. 

Chapter 5 introduces the concept of "depreciation 
factors" for the purpose of evaluating strategies fron 
positions other than tne first one in the r",£-*ne. A oleusicle 
arrmment is £iven for the predominance of LSTRAT over 
HCTKAT in TSI. Modifications of HSTRAT and B5TRAT are 
sup^ested, usin^ these depreciation factors, and evaluated 
by Monte Carlo methods so that they can be compared to the 
original strategies ■ 

Chapter 6 introduces variants on Tree Solitaire 
as a means of suc^estin/r ways of extending results a ready 
obtained. In particular a game which seems a much more 
reasonable idealization of **ames like chess is introduced, 
and a start is made toward applying the methods and results 
of the previous chapters to it. 
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CHAPTrlR 1 
TREK SOLITAIRE AND ITS GAMS TREK 

Host rgjne playing programs usinn locK- ahead abstract 
n ^.anio tree from the rules of the f*ame f an*l assign values to 
the points in the tree according tc an evaluation function 
defined on the physical configurations of the corresponding 
nasitions In the f*fi3B. In studying tn r : mrftlruts Of thO 
jsort-ahead T}rocerV:re theoretical ly* or*3 is most interested in 
, T ust Lhe ^ane tree itself and the values assirned to its 
points. Therefore in tnis report the orocctfure is to generate 
the ffame tree, asairm values to its points, olid then to define, 
with as much simplicity as possible, a game which tne tree 
represents. 

Most consideration is Riven to a one-person r 7 ome f or 
actually class of one-person games of this type, which will be 
called "Tree Solitaire" (TS), and has been developed from an 
idea proposed by Seymour Papert, Variants on this ftame will 
be discussed in Chapter 6. Trees for TS are binary, end 
generated as follows: 

Start with an initial "fork" as shown in Fir^re 1,1 



1. In unpublisned notes and conversation. 
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Tfie top point of the form will be called a st>lit print 1 

Top or "split* Pojnt 



* - 




Fipuro i,ii a Corn 

The bottom points 01 tne fork are called the Innaefliate 
"successors" of the? ton point, find somo^i^es "brencfcis. 
The bottom points of the initial for* may be either I'jrther 
"split points'* or simply end points. Whether a bottom point 
is a split point or an endpoint is decided by some probabilist 
method, chosen so that the probability that the point is sn 
endpoint is a /*iven number E- The probability that it is a 
split point is S = 1-E. Whenever no confusion will arise, 
an endpoint will be referred to as an E, a split ooint as an 
S. If a bottom point to a fork is determined to be an 3, a 
new fork is added to the tree with this point as its top point 
and two new bottom points must be dealt vfith. As soon as all 
available bottom points have been determined to be E's, the 
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;-onoration C a binary tree is complete Figure 1.2 gives 
an example. 




Figure l,i: A x'tneratej tree 

The tree will be ~snara±ed witii a finite number oi 
points if E > -5. 1 The need for this will occasionally be 
apparent, but many of the results to be obtained will be 
valid for E < . 5, and this will be explained when it occuro. 

Assign to the top point of the tree the value 1. In 
some probabilistic way* decide on two number whose sum is 
thiG top value, and assign these two numbers to be the value; 
of the branches of the fork. Thus In Figure 1*1, the ton 
point has value 1 and the bottom points mi£ht have values .6 



1. Harris, The Theory of Branchi ng Processes , p 



• i 



- 9 - 



and . . **. For any forK, the ratio of the value of the higher 
valued branch to the top value is called the "split ratio". 
The split ratio here would be .6. If a bottom point is car .- 
Si we split Its value between its two branches in the sane 
probabilistic way. This process is continued until Ml 
points of the treo have been assigned values. In a finite 
tree the sum or trie end points will equal 1, 

In order to construct a ftnme, somo of these nnd- 
polnts will bo D*jietJ wins, the rftot i-oaisec. There will be 
no 4raH3* T*rti ci:cil?ir ~<?lnodc of asslrninr wins and losses 
£t7fi rise to two different forms o:' Tree Solitaire, 

The dependent form (TSD) treats the values of the 
endpoints as a probability distribution, and assinnn one *rfn 
to the Rome, locating it at an endpoint chosen according to 
the distribution. All other endpoints are losses. Thus the 
value of an endpoint is the probability that it is a win. 
The reason that this form of the came is called dependent is 
that, if one endpoint of value A is known to be a loss, 
then the probability that any other endpoint is a win is its 
original value divided by (1-A), since we know that one of 
the endpoints must be a win. 

In the independent form of the fiarae (TSt), the /alue 
of an endpoint is used as the probability that the endpoint 
is a win , the decision bein^ made 1 n d e pendent 1 y for each 
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endpoint. Thus there need not bo exactly onn win. There 
may be zero, one, two, throe, etc. Knowing that one endpoint 
Is a loss rives no information as to the others. This form 
of the rrame was introduced because it seemo closer to real 
^anes than TED, and a" so because it w/v: erroneously thournt 
that TSQ would be too como'icated to analyse. (See civittter 

To complete the definition of Tree Solitaire, one 
must 07r])i«ln cm raloB ot pis:/, Clear":/, i' the ^km tr&G 
in to wor£ o£ RftflK? trios should, a "play"' of tho roue will bc- 
reoresentable as o. n&ih froci th'j top point to an end ooint, 
with the noints of the oath representing oositions in the f;&jnc, 
and the lines between thSrt NpMMntlJ&fl; lcRttl no-es. Tnis Is 
exactly what is done. 

The 'position" correspond inn; to e^ch point is a certain 
set of information about that part of the tree: 

1. The value of the point Itself 

2. Whether it is an E or on 5 

3. If an E, whether It is a win or loss 

H. If an E, the values of its two branches, or 
equivalently, its split ratio. 

A move is made from a position corresponding to on S 
point to a position corresponding to one of its branches. A 
binary choice is Involved and herein lies the opportunity for 
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strategy. The ruiun starts in the position corresponding; to 
the too noint of the tree and continues until the position 
corresponding to an endpoint is reached, and is won if find 
only if that point is a win* From now on 'the position 
corresponding to" will be omitted and positions will bo 
identified With the corresponding points, hopefully without 
loss of clarity. 

It would be apropos txere to e:mloin how this name 
cwsparoa to wore Coflp'iiti-itwl once 11 :o chcu^, ml to Justify 
it-" use as a starting nolnt in the- stud;/ or tho problem of 
loo^-ahead. Pir3t Of nil, the simplification to a one-pornon 
ftfitte is reasonable in the beginning of such a study, as well 
as convenient due to tne nature of the ^ene tree and its 
valuation. Similarly, having the tree be binary and thus 
having only two possible moves from any position, seems a 
reasonable simplification, as does the fact that an ordinary 
"play" of the j*ame will only comprise a small number of moves. 
Letting the va ; ue of a position be its probability of beinr; a 
win is a straightforward valuation, even if it is perhaps more 
powerful and simpler than those used in chess playing programs. 
In TS the chance of any position beiny; a final one is a 
constant E throughout the f^ame and this may give rise to 
doubts, but is perhaps justifiable by the fact that in 3ames 
like chess a number of the moves from any position are such 
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outright blunders as would create a Joot !*;omc for the mo;cr — 
and a win for his opponent. 

The principle objection to T3 must arise from trie 
fact that the values of positions of necessity decrease as the 
piay of the ^ame procedes farther down the tree. This Is a 
reasonable criticism, and the v/eahness rererred to does h&ve 
an appreciable effect on the results, as will be seen when 
they are comnared with those for a variant class of ponies 
Without thi:: Wtftt-tness, in Chapter 6. 

However, Tree 'Jolitaira is a nice ?nowcase for tne 
". arious possible liraitod looh-ahead strategies, and ftllows 
much straightforward analysis of them. It can serve as a 
worKshop for de-eloping methods of evaluating strategies* and 
it as well has a certain amount of intellectual appeal all by 
itself. 
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CHAPTER 2 
LOOK-AHEAD STRATEGIES AND THEIR EVALUATION 



- 



In the context of Tree Solitaire, a " look-ahead will 
be definod as seeing a successor position to the one you n.re 
in> or nave already seen, tey "seein;; a position" will be 
meant the acquisition of the entire set of information asso- 
ciated with the position, as defined in Chapter 1. The 
difference between a ''look-ahe&d*' an:! a "oovo" is that lectin: 
ahead to a position, although it Rives you as much infcrr:a*ion 
es moving there would, does not commit you to that position, 
whereas moving there would 

A look-ahead strategy will work in the following way. 
Suppose you find yourself in position X. In accord with a 
given rule you choose those succeeding positions at which you 
will look -- the rule may allow you to use the result of your 
earlier look-aheads in choosing where to use your later looks. 
Once the look-aheads have been completed, you make use of the 
information to decide according to a second rule where to move, 
thus arriving at a new position, where, if it is not the end 
of the game, the process is repeated. Any information you may 
have gained in previous look-aheads as to the successors of 
this new position is assumed to be forgotten. 
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A look-ahead strategy can thus be seen to be defined 
by Riving the two rules that govern it: the one that tells 
where to look, and the one that tells where to move, rtiven 
the results of the look-ahead, 

Strategies may be evaluated according to a number of 
criteria. Hi^h on the list, however, must be their effec- 
tiveness at achieving a win, over the whole cl*ss of paries of 
Tree Solitnire. The principles of such evaluation are 
demonstrated in the following two c&ses of strate^i^s 
involving no loo>-ahead at all, 

HOSTRAT is defined by the rule that moves are chosen 
completely randomly, without any reference to the values of 
positions. In this case the expected value of tne- position 
you move to will be 1/? the value of the position you ore in 
The chance of winning on a r,iven move is the expected value of 
the position at the end of that move times the chance of 
getting that far in the ftame, under the p;iven strategy * times 
the chance of the position being an endpoint, i,e,, times E. 
The chance of winning the game at all thus becomes a conver- 
gent infinite sail. In this case. 




E/(2-S> = E/(?-(l-E)) = E/(E+1) 
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If E -~- -J , then the probability of winning W . = .;i'i33- 

NOSTRAT is not the best possible strategy involving 
no look-oheads. That honor belongs to the obvious stratory 
of always moving t r / tho higher valued or Lne two branches of 
a fork> which stratd^y will be called simoiy r/TRAT. To 
calculate tne chance of winning under STH/*T» we make use of 
the expected va*ue A of the split ratio as a prediction 
for the way values split at every split point we encounter. 
It 1 the loo point of a fork nas value X, the expected va&vn 
of the hi£h branch's value is AX and for the lower **alued 
branch EX, where B * (1-A). Since the value at the top of 
the tree is 1 ( the cnance of winning under GTRAT can be 
seen to be 

L*~t ft5 > ~ T^AS " 1-A+AE = S?AE 
n=o 

If K- 1 , A « ^ , then the probability of winning 
W = .6000, 

Against "effectiveness" must be weighed the "cost" of 
the strategy, in terms of, say. computer time. Since it may 
be lssumed m tne case of look-ahead strategies that the 
principle time consumlm* factor would be the number of loofc- 
aheads, cost considerations would lead us to want to minimize 
this number per move over the class of all /same trees. 
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However, in tryinr, to balance this against the importance 
of having an effective strategy, any theoretical treatment 
would involve a certain amount of arbitrariness, which mlrrht 
have a considerable effect on the results. In order to avoid 
this problem, tnls study has concentrated on comparing as to 
tnoir effectiveness various strategies whose costs *ire the 
Same. '* Limited look-ahead" strategies involving one looK- 
*ttie*rf Will f;rst b^ ftxaHined, and then thnan Jn^olvinr; two 
ana possibly mora. 

But firsts for tne sake of perspective, we will note 
a result at the other end of the spectrum — unlimited look- 
ahead. With unlimited look-ahead a win is assured {in TSD; 
in TSI a win is assured if there in fact ^s one). So wnat 
we must do here is pick from a number o^ equally "effective** 
strategies, that which is least costly. Such strategies have 
irPffoniTion the fact that they use their ]ook-ahctd to find the 
endpoint which is a win (in TSD; in TSI, such an endpoint 
if it exists)/ and then the proper choice of moves is a matter 
of course. 

Intuitively, the best of these look-ahead strategies 
would be the one which looks at the highest valued of the 
points open to look-ahead i.e., the points which are successors 
to the points already seen. Thus in Figure 2,1 you would first 
look at ,6, then ,5, then .«, .4, .3* >2, .1. 
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Figure P. 1j A s&ttple 3 an* tree with sample uftluw* 

Papert" proved that this is indeed the best method. 
Proof: First note that since any method of selecting what 
point to look at next is independent of whether that point is 
an E or not, over the class of all Rama trees the proportion 
of E's looked at to the total number of points looked at 
should be tho same, that is, should be the number E. Thus 
to minimize the total number of positions looked at, over the 
class of all p.ame trees, one need only minimize the number of 
endpoints looked at. 



1. In unpublished notes. 
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The expected number of endpoints to be looked At in 
order to find a win is, for any strategy in frames of TSI: 

A^i + (i-X^A^a + ... + (i-A x )(i-A 2 ):.-. (i-^jvr 

nnd for TSD 

A^l + Ag-a + ... + A n -n 

where Aj Is the value of the i*" endpoint looked at, and 
n is the number of endpoints. The 0-A*) Arc cancelled in 
the formula for TSD because, if A- is not a win, then the 
chance that A* +1 is a win is Increased to A, ,/(l-Aj). 

In both cases, the sum will be minimized if the end- 
points are ordered so that A, > Ag > , . - > A^ which cen only 
be guaranteed by the strategy under consideration, Q. E.D, 

Wc now return to strategies involving a fixed number 
of look-aheads. The present chapter will conclude with a 
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description of the strategics to be evaluated in the next two 

A 

chapters. However, first it will be noted that unlike the 
stratep^y .just described* they are all what will be called 
"ordinal" strategies. An ordinal strategy is one which does 
not make use of the precise values of the two branches of a 
fork, but only notes which one is larger. Only such "ordinal*' 
information is used in deciding where to look next, as well as 
choosing where to move next. The use of the precise values* 
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either to choose the next move, or to choose where to loci; 
next, at first thourJit would seem quite valuable* however 
the actual advantage in the case of one or two, look-aheadc 
may be minute, end there is a corresponding increase in tno 
amount of onlculntion needed ond hence time used. TfleOC .'aid 
oth^r questions nrfl discussed In chapter 5. 
The strategies; 

:l": J *.Y. r *ri'* ;*.': -*>e*W* LrAfc at the hi^har br'iri^h. If Jt 
iji on :'•, stcvo tner': P If it is on E nnd a win, move there. 
Otherwise move to the tower branch, 

LSTRAT. One look-ahead. Look at the !ower branch. If it is 
an E and a win, tdovc tnere. Otherwise nave to the hlrjwr 
branch. 

&MSTRAT. Two look-ahends ( or less). LooK at the hi^h branch, 
If it is en S, look at its hi<*h branchy If either of these 
two is a win, or if both are S's, move to the high branch. 
Otherwise, move to the low branch. 

2LSTRAT. Two look-aheads (or less)- Look at the low branch. 
If it is an S, look at its high branch. If either of these 
two Is a win, move to the low branch. Otherwise move to hi^h 
branch. 
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BSTRAT. Two looK-oheads # Look at both branches. If cither 
is a win, move there. Otherwise, move to the highest of tho 
two which is an 5. If both are losses, accept defeat with 

a smile. 

. . * - ii 

These; five exhaust the possibilities for one and ^wo 
looK-ahead ordinal str;ite?;ics, in the sense that any other 
such strategy can be shown to be obviously inTerlor to at 
j east one of these wnich agrees with it as to number of icofc- 
aneads, whereas the r*nKing of these is not at first obvious. 
Tne next two chaoters try to discover this ranking. Althou*n 
the results In the present chapter hold equally well for both 
T3I) and TGI t the treatment is different when looK-aheads are 
involved, and so one chapter is devoted to each type of r.ame 
separately. 
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CHAPTER 3 
ONE LOOK-AHEAD STRATEGIES IN THE DEPENDENT FORM 
OF TREE SOLITAIRE 

The results in this *uvi the following chapter will 
be proved for the restricted class of r^ame trees witn 
constant split ratio A, .5 < A £ 1.0, It is o Dlausibjs 
n^n Jr 3ct*jro tn^t tho results ore a nnrox i mutely true over the 
elau of s! 1 rule trees* zo Ion; 1 ; as A ir tho ey P pcct*vt 
value o!' trie spl i t ratio. 

In tnis chapter it will be proved that In T3D» 
LSTRAT is more effective than H3TRAT for all allowable 
values of A and R, Is this a surprising result? With 
unlimited look-ahead the least costly strategy that insures 
a win involves looking at the highest valued position first; 
with one look-ahead the most effective strategy is to looK 
at the lowest. But this is no contradiction. Also, vie must 
remember the whole strategy, not Just the look-ahead rule. 
LSTRAT, by looking at the smaller branch, insures that if 
there is a win at either branch on this move, you will ROt it. 
If this is the virtue of bein** "aegrassiv©", then HSTRAT 
has the advantage of '"cautious," for it assures you of not 
losing on the current move unless both choices are losses. 
It is difficult to see which factor outweighs the other, so 
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we turn now to the actual evaluation of Lh2 strategies. 

Now that look-aheads have been Invoked, It seems 
unlikely that the probability of winning under a Riven 
strnte^y will be calculable In as simple a manner as was 
us^d in chapter T\ The use of a slrvle exncctod value for 
tfti value of the nocition on u /<;i.'sen move (at a ~iven ply of 
the <*ame treej will lead into difficulty. With looK-ahead 
stratfiftioSj the orob^Mllty distribution of position valuer* 
ot a j'.i\en ply is dependent on the probability listribution 
for thn ore/lous ply. This 'news that the expected value 
for the position value at one ply is not a direct function 
of the expected value for the previous ply. but of tne entire 
distribution of position values. Thus we cannot, as wan done 
for :;TRAT, have a single estimate of the position value at 
one my, and use that to calculate our estimate for the next. 
Instead we must find a way to retain distribution information. 

For 3 tunes with o constant split ratio this is not too 
hard, and hopefully the probabilities of winning thus obtained 
will be £ood approximations to the probabilities for the class 
of all games whose expected split ratio is equal to that 
constant. 

With constant split ratio, the number of possible posi* 
tion values at the n " ply (with the top point of the fta^e 
tree beinK ply 1J is precisely n. For example, with split 
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ratio A and B ~ 1-A, the possible position value at tne 
first ply is 1 , at the second ply the values are A an'l B, 
at the third A * AE, and B , ani so on. To «*valuatn thn 
chance of winning for a ftlven E and A under a particular 
Vtrate-'y we sun, for oach oly, the product of each possible 
position vr.iue times t,he probability that such t position is 
on endooint times the probability, over the class of all r.ame 
tr^cn with split ratio A and enriinr: nrot^bility E# that 
yo:i Could wit to such r -i position at the given nly ir.ier ~r.e 
^irnte^;;, Tnerv wo s^n tne values thus obtained Tor the 
probabilities of winning at each ply o/er all plies from 

ply '*? on. (This 1e the same as summing the chances of winning 
on tne first move, second move, etc) The total is the 
probability of winning at all, under the niven strate:cy over 
the class of all r.ames with the particular split ratio and 
ending probability. The sum will converge, as it is clearly 
bounded by 1. 

To aid in this calculation, it is convenient to have 
a diagram as in Figure 3.1 for HSTRAT. The diagram is only 
partially drawn here. However, it is in a sense recursive, 
and once one understands how it is generated, one does not need 
to see the rest. As such diagrams are much relied upon in 
this report, a detailed explanation of this one will be riven 
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here, so that the reader can generalize and thus understand 
later diagrams without much further explaining. 




Figure 3,1; Evaluation Diagram for HSTRAT 

The points and lines have numbers associated with 
them. The points represent possible positions you may be in 
or looKinp, at, and the circled numbers are their values. 
Lines are directed from left to right or from top down, and 
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the numbers >»ssoclnted with them are trie probabilities or 
going from the st^rtim*; point to the endpaint (in the case 

- -Of- HSTRAT, -of -jrolnr; from loo king at- thanifth branch to 

mo ving, to the low branch, or of coinr, from the top point of 
a fork to loo king at its high branch). Th* first horizontal 
row of points represents the first move (or second pi?/), *nd 
so on. The probability of being in a ^i V en position (i.e., 
at a r.ivftn noint oi' the diaj*rajn} on a riven move (i.e., at a 
rivon nly of the ;^n;.ie ~-rr;ej is calculated recursively by 
acGociatin^ to eacn noint in the dia^rnjn :t second number (its 
"nrobobility" ) enual to the product of this number for the 
point's irwiediate predecessor, with the number on the Jine 
connecting the two, (In some later diagrams a point may nave 
more than one such predecessor, in which case we taJce the sum 
of such products.) The first point in the tree has 1.00 for 
its probability. 

This particular diagram is rather simple. For instance, 
on the first move the higher branch has value A. You move 
their unless It is a loss, the chance of which is E(l-A). 
If it is a loss, all other probabilities have to be divided 
by (1-AJ because we're in the dependent form of the tiasne, 
and 30 the lower branch, to which you move, now has value 
}\li\ - 1* and is so labeled. Essentially this same kind of 
process Roes on at each move. 
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Now for the evaluation Itself. The chance of winning 
on the first move is 






EA + E(l-A)-E-1 - AE + BE* 

where B - (I-/). On the second move it in 

2f?R + Sfl-A^B * " B a + (I-AJeVjA +■ tl-AJ ? E 3 3 

(l-A*) 

(rennrnher - L - - 1-Ej ■ and so on, Howe 'er. cnncollin: the 

(WO and simplify In.-;, tne second term is 

(SA * BES)(AE f BE y ) 

or (3A + BES) times the chance of winninK on the first move. 
This turns out lo be true in general, i. e. , study of the diagram 
and the wsy the numbers in it are assigned shows that 

L&I4MA: Tne chance of winning on any niven move (except the 
first) is (SA + BES) times the chance of winning on the 
previous move 

This means we have a geometric series as before, and so we 
have proved 

THEOREM 3.1: Over the class of all Rames of TSD with 
constant split ratio A and ending probability E, the 
probability of winning under HSTRAT is 

■ 
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rt 



M£"VU V 



w.crc B = 1-A, 



1-S. 



For example. If B-1/2. A = 3/*. then W„ - 7/9 - .7778. ■ 
Similar r.ood things happen for L3TRAT in TSD. The 
diagram is Fir.ure S.8, where the possible positions ftt each 
Tiiy are located in narrow bands of not ouite colinear points. 




Fijjure 3.2: EvfJuatlon diagram lor LSTRAT 
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This diajrrom is explained as follows* The first thins* you 
do is look at the iower /alued branch of the initial fork. 
8*19.8*. value is, B. V._ it is a win, you move there. If it is 
a SDlSt ooint, which happens with probability 5, you move 
to the higher v+ a*Jued branch, whose value is A. If Jt is a 
loss (probability E{l-B)), you still move to the Utihotf 
branch, but now the probabilities of winning are the ori-Jnal 
values divided by (1-Bj so tne value of the higher branch 
is A"(l-B) = A/A ' i* In a sJxilar way the r r :s*j of tnc 
dia^rar: :aay be generated. 

The chance of winnin;; on the first move Is 

HE + S£A + E(!-B)E-1 = EB + ESA + E*A 

On the next move, the chance of winning is 

ES 3 A 2 + ES ? AB + E 2 S ? A 2 + ES 2 A 2 + Z 2 Sfl3 + E 3 SA ? 



or 



{3 2 A + ESA)(EB + ESA +■ E S A) 



The reader may verify for himself that a/^ain we have 
a ratio which holds from move to move throughout the p;ame. 
The sum of the resulting geometric series is 

EB+E3A+E 2 A E(B*SA+ ( 1-S)A) E 

1-?A + ESA 1-(1-E)SA-ESA " T^S* 
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and y*e have shown 

THKONKM "J,?: Over the class of all names or TSD with 
constant split' ratio a ond ending probability E, the 
probability of winning under L3TRAT is 

K. = c'(l-SA) 



"L 



where S = l-E t 



if k :. :- wtJ a * J ii tnon ar L - 1/5 * .Sooo. 

How we oxc at :a:;t nrepfcred to Drove 

THEOREM :;. i; Over the class of all cornea Of T:;D with 
constant split ratio A and ending Drobability K, for any 
combination of values of E and A, ,5 < k < 1* " and 
.5 < A < 1., both L.7TRAT and H3TRAT are more effective 
than STRAT (no look ahead), and LSTRAT is more effective 
than H3TRAT. 

Proof: This result is obtained by simply comparing 

w - itsa < for STRWr >' w h - T?4fraiy - ^ d h l - dk 

The first is clearly smaller than either of the second two, 
so all that need be shown is that W_ > W„ , which will be 
true if 
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That will be true If 

1 - S(A+BE) - (A+BK){1-5A) > 
i.e. , if 1 - SA - SB£ - A - BE + SA** + BESA > 
i.e.. if 1 - SA - SB(l-S) - A - B(1-^J + :y/ + iAB(l-S) > 
i.a., if ] - B(A+B) + s'B - (A-t-BJ + SB + oA(AfB) - S^AB > 
l,o.. ir 1 - 3 + E :: B(]-A) -It 3(A+Bj >0 
1.*. , iV r/V > G 

But this lost is obviously true for ,5 < E ** 1« an* 1 

.b < A < 1, Hence W. > w„ and LSTftAT is the more effective 

It n 

strater^. Q.E.D. 
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CHAPTER 4 
STRATEGICS IN THE INDEPENDENT FORM OF TREE SOLITAIRE 

The neatness or the formulas for W. and W H in 
T::i) is due to an inherent cancellation which 'Iocs not take 
nlace there Tor tne tm-laoh-anead strategies, go the treat- 
ment of these has been postponed to this chapter on TSI. 
In the case of this class of wnes, no clo-ed formula nas 
yet been difinavered for the probability of winning under any 
of tne strategies involving Jook-ahead. 

However, diagrams such as tho::e in the preceding 
chanter can stjll be constructed, and sup-ost the ideas for 
recursive computer programs which can calculate the appro- 
priate probabilities of winning (W H> tfjj and by analogy, 

W 2H' W 2L J and W B^ move by move. This process, it turns out, 
need only be carried on for the first ten moves to rcet 5-place 
accuracy, although occasionally a small correction factor must 
be added to taHe care of the chance of winning after the 10 tn 
move, 

A separate pro^rajn has been written in HAD for each 
of the five strategies, in addition to one for two limiting 
cases which will be discussed below. Each program when Riven 
values for E and A outputs a sequence D(l), D(£), . . . ,D(10) 
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of numbers representing the probabilities or winning on the 
ist, 2nd, etc. move. An output program then prints them and 
the ratio between succeeding terms, as wen as their 3um and 
any correction factor. The diagrams and programs are 
oresented in Appendix A. With the exception of 2H2TRAT 
and 2LSTRAT, all these proprans were checked against nand 
calculations. Those two were too complicated lor hand 
calculation, but the internal workings of both programs were 
thoroughly rone over and f^und satisfactory. 

In th? esse or the one look-ahead rrtrafco^ies, the 
conclusions of the Preceding chanter Been to carry over to 
TCI, even If their proof does not. Table 4« 1 r;ive the calcu- 
lated values for tf*» W„ (and tf) for 'arious combinations of 
E and A. 

As can be seen, W L > W H for all E and A Riven 
here, end both W, and H„ are greater than W. For fixed 
E, M H (and indeed W) approach W. from below as A 
approaches 1, In the limiting case they would all be equal, 
as all the value of the top point of a fork would RO to its 
high branch. Similarly, for fixed A, « H (but not w) 
approaches W r from below as E approaches 1, ap;ain with 
equality for the limiting case, because then both bottom 
points of the initial fork would be endpoints. Eoth these 
conclusions could have been made for TSD, because the numerator 
of the fraction representing W L - W H was (1-E) a (l-A) « 
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A 



W 



'■-, 



M 



H 



*- _ _ 






T 




.5 


5 


. 33333 


. »15ft 


.UbllO 


-5 


66667 


.5 


. 61029 


. r j')'y.H 


.5 


7*i 




. 69763 


. 66101? 


.5 


9" 


i 6x8 18 


.83914 


.^3266 


. c . 


•>5 


,'M78 


. VK.05 


.•--89" 




* * 


. •.-: .:■. 


. 53' V> 


.' 8: .;■. 


.5 


.y r .' 


.7-8^-' 


. && ■ 


. ■•".*,. 


.5 


5 


. 33332 


. 55152 


.48110 


-75 


.5 


. 49857 


. 6630C- 


. 64285 


-9 


.5 


.47363 


.72041 


.71253 


GO 


.5 


.4^74? 


.74717 


.74653 


.25 


.75 


. 4?70o 


.31847 


. i»6770 


• 5 


.75 


.6 


. 68768 


.66102 


.75 


-75 


. 69231 


. 76722 


.75745 


o 


.75 


.72973 


.79693 


.79371 



Table 4.1 



ComDutcr results: evaluation of probability 

or winning under STRAT, LSTRAT, and H3TRAT 

over the class of games of TSI with constant 

split ratio. 
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Tne case in the table with E < ,b can be included 
because in TSI a ^ojne can be played without peneratin^ the 
entire ^ome tree. All that need be venerated are the positions 
that vrill actually bo looked at or mo *od to under the riven 
strnter'y. (This is how computer simulation of the r.ame Gro- 
ceries in the Monte Carlo method of evaluation to be discussed 
below, j The expected length of a patn in the tree is only 
I "E» co by such limited feneration we can reduce our work to 
ron&onftble amounts as Ion:: as 5 iz not too small. 

Actually.. two factors -Hsturb fchic Mtfmate of rwm 
len;th when we are denliru* with strete-iec involving loDfe-flhead, 
EndpointE which are losses may be foreseen and avoided. On the 
other harrfj endpoints which are wins nay bo foreseen an't 
purposely sought. The results of actual simulation of LSTRAT 
with E * .5 and E ■ . ?5 show that playing a Riven amount 
of frames for the smaller E takes only about 1,4 times ns 
lon^ as for the larger E. The factor for BSTRAT is about 

2.3* reflecting perhaps the greater ability of that two- look- 

■ 

ahead strategy to avoid losing endpoints. 

Since computer simulation of game playing has been 
introduced, it would perhaps be appropriate here to present 
the results of an attempt to estimate W* and V/„ in a 
special case, by Monte Carlo methods. E was set to . 5» A 
to .75- Actual flames were played under LSTRAT and HSTRAT, 
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with the q&ne' tree being generated as needed with the aifi of 
a MAD random nuraber generator. 20*000 or more names were 
Dlayed to obtain the estimates. A '"trial" was a series oi 
30,000 j-a/nes. The variation between trials Tor any particular 
estimate was at most .000^0, Indicating considerable 
stability. The results are summarized in Tab 1*3 J|,B, 



Diagram Method Constant 3p3.it Ratio Variable Solit Ratio 

I II 1 II 

tf Ti . 68768 , 67090 . 6y/;> . GSR40 . GK>10 



J U 



, 66102 . 65V ~J0 . 66640 . 67070 . 65950 



Table **.£ Monte Carlo estimates of W L and W„ for E =■ .5, 
A - .75. 



This table. In addition to comparing the Monte Carlo 
estimates for constant split ratio with the value obtained by 
diagram method In Table 4.1, j^ives the estimates for W T and 
W„ when the split ratio is not constant, but obeys a probability 
distribution such that every value between .5 and 1.0 i3 
equally oossible (and hence the expected value of the split 
ratio is ,75)* Two different value (I and II) for each 
split ratio alternative are given, because it was found that 
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different utilizations of the random number generator «,ave 

rise to si/mificantly different estimates. Under utilization 

I, low values of the random number caused wins. Under 

utilization II, hirtfi values caused wins. 

The difference caused by this effect is not the s*:ne 

for both strategies, or for both split ratio possibilities. 

Under constant split ratio, the difference for W- is ,01100. 

for W H .00710. Under variable split ratio, the difference 

for h. is ,00570, for W„ .011*0, All those are consi- 
st n 

durably greater than the .000*0 marp:;n o:" error in the 
individual table entries, thus indicating that these differences 
are significant, and, although the table 4,1 values do lie 
between tne I and II values for constant split ratio under 
both strategies, casting a doubt on the use of Monte Carlo 
methods for precise estimation of probabilities of winning, 
at least with the present random number generator. However, 
such methods do, at least here, preserve the ranking of the 
strategies, and it can be noted that apparently, the ^Iven 
non-constant distribution of split ratios here does yield a 
greater probability of winning under both strategies than for 
constant split ratio, even though the expected value for the 
ratio in each case is the same. An example of a Monte Carlo 



1. Time pressures have prevented the devising of a random number 
generator which would be for our purposes less biased. 

■ ■ 
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program is listed in Appendix B. 

Returning to the diagram method of evaluating 
strategies ( and the assumption of constant solit ratio, it 
Is interesting to note how the chances of winning on naxti- 
*:uJs.r moves (i.e., at particular ply in thr: r»ane tree) 
co.T.p.ire for K.TTT'.AT ->nd UVTRAT. Table 1."- i- a. listing oT 
Dfl> thr';u«n D(10), as defined above for the two, with 
£ .5 wvl A -- .V*, 



Move 

1 
2 

3 
U 

5 
6 
7 

a 
9 

10 

SUM 
CORRECTION FACTOR 



W L and W H 



L3TRAT 


rnnuT 


.45311 


. 3?06;i 


. 15? 53 


.15733 


. 052&3 


. 06507 


.01867 


.OP 7 35 


. 00670 


.01161 


.0024i 


. 00500 


.00089 


.00216 


. 00033 


. 00094 


. 0001? 


.00011 


. 00004 


.00013 


. 68765. 


. 66038 


. 00003 


.00011 



.68768 



.66102 



■Table *i,3 



Chances of winning under HSTRAT and LSTRAT 
for the first 10 moves* with E •* .5, A ■ .75. 



A _ - 



- -1 
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The ratio between successive term;; for I5TKAT 
i 

begins at . 3'i66l ond increases to # Yfl>8 by the 10 th 
move. For HSTRAT tne ratios t*o up j ror* ,40jpW to ,*'j57^* 
This leads to the fact tnat here LSTHAT's advantage Is 
entirely contained in the first move (For other E and A< 

it is sometimes contained in the first two. ) Tnen tne 

• 

cumulative effect ol cautiousness'' begins to pay off. The 
fact that th<* chance of winnine on the 6 tft move is twice as 
rwih under KZTJUS %B under LOTHAT is duo to the eoaulatlYC 
etfoct of novinB need that strategy on the flr:;l ^ moves, thus 
reducing the chance of winninr early in tlie came, rather than 
to any inherent superiority of H3TRAT once you ore that far 
alonr, in the fcame and position values are small. This will 
be shown as a byproduct of the work in the next chapter. 

Now turning to strategies with more than one look-ahead, 
we note that in a way 2HSTRAT is an extension of HSTRAT, 
One could easily continue this extension to a 3HSTRAT, etc. , 
with each new strater^y a slight improvement over the others. 
For instance, in Fifpjre **,1, 2HSTRAT would move to the A 
position, then notice the loss at A J and move to AB* However 
3HSTRAT would see the loss right away and move to B, which is 
a greater value than AB, 
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LOSS 



Pirure ^.1: A possible frame tree, labeled with the values 
of the points. 

So we have the general concept of XHSTRAT,- where 
X * ],3»3.*» In the limit we obtain an unlimited look-ahead 
strategy »H£TRAT whose effectiveness serves as a limit for 
all cuon strategies. *HSTRAT looks down the high branch of 
a fork, and i ts hir t h branch, if it splits, and then its hij*h 
branch, etc. , until an endpoint is reached. If it is a win, 
you move to the original high branch, if a loss, move to the 
low branch. {A very inefficient way to use unlimited look- 
ahead, even though it agrees with the one discussed in chapter 
2 on the first ? or 3 looks, depending on A. Also inefficient 
due to the repeating of previous look-aheads after each move, 
but this is a common fault of all XHSTRAT's). 



Similarly, an «LSTRAT may be defined as a limit of 
XLSTRAT's. Table 4.4 gives the evaluations of these, and the 
already defined two look-ahead strate-ies, for various E 
and A. 



'B 



W, 



2h 



W 



-L 



tf 



?H 



W 



iH 



.5 


.5 


.61335 


. 60013 


.6l58i 


. 5*025 


.63 583 


.5 


. 56667 


. Wo 


,6684'j 


.6066*: 


,6i-..?;5 


. 64563 


. ,: 


.73 


.TLVtf 


.71673 


♦73324 


.67301 


. 69'.€7 


-5 


.'• 


.c.4587 


.6*783 


.3546-' 


.833B0 


.''••3551 


.5 


.95 


,9i?y7 


. yi 375 


. 916J"j 


.00915 


. '(0'yUl 


.5 


.99 


.^058 


. '£063 


.98076 


. «^80 39 


.98040 


.25 


.76 


. 54486 


- 5^35'' 


.61633 


.47841 


. 5^694 


.5 


.75 


.71436 


.71673 


.733a* 


. 67301 


.68967 


.75 


-75 


.78183 


. 78o?4 


.78183 


.76470 


.767?P 



Table t,4: Probabilities of winninc under BSTRAT, ?LSTRAT, 
*LSTRAT, 2HSTRAT, and -HSTRAT, over the class of 
Frames of TSI with constant split ratio. 



For A = .5- 
reduce to the 3ajue thin;** 



W r * W u because 



•HSTRAT and •! SIR AT 



w 



*L 



would not equal Wy„ for 



A * .5 because they would differ on where to move if the last 
point looked-ahead to were a split point. 
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A much more interesting table can bn constructed 
by using: Tables 4,1 and 4,4 to rank nil the strategies as 
to effectiveness: 

E a 1 3 3 4 5 6 7 



.5 


.5 


-L— tie-— H 


B 


2L 


L 


i>H 


H 


.5 


.66667 


-L 


B 


2L 


-H 


L 


2H 


H 


.5 


-75 


•L 


2L 


B 


-H 


L 


2H 


H 


e 


it 


«L 


2L 


B 


L 


-H 


2H 


H 


-5 


.9b 


•1. 


?.L 


B 


L 


-H 


2H 


H 


.5 


.99 


-L 


2L 


B 


L 


-H 


BU- 


"tie"— K 


.^5 


.75 


-L 


. ?L 


B 


-H 


L 


SH 


H 


.5 


.75 


•L 


PL 


B 


-H 


L 


2H 


H 


.75 


.75 


■L 


B 


2L 


-H— "tie" 


--L 


2H 


H 



Table 4,4: Ranking of strategies as to probability of winning 
over the class of ^ames of TSI with constant 
split ratio. 

Table 4,5 speaks for itself. It bears out the con- 
clusion that \t~ > W„ H . However It also shows that it is 
not true that in all cases XLSTRAT is the best X look- 
ahead strate/jy; For E = .5 and A = .5 and A - .66667 
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W_ > W_ T , The wastefulness of the XHSTR/T's "caution is 
seen from the fact that one look-ahead LSTRAT is better 
than PH5TRAT, and even> in certain cases, better than 
»HSTRAT. 

However* rigorous mathematical Justifications for such 
conclusions are yet to be discovered. One of the purposes of 
the next chapter is to p.et a start in tnat direction. 

As one 1 ar*t way of looking at the results of this 
cn^nter, let us note exactly rtavr much thn jce of look-ahr-ad 
in ordinal looK-ahoad strategies imnro *es the chance of 
vrJnninr.. Table **.5 ^ives for various combinations of values 
for E and A the ratios of the probabilities of winning 
under the most effoctive one and two look~aheo/i strategies, 
to the probability of winning with no look-ahead (under STRAT) 



K 


A 


V 


one look-ahead 


two looi-.-irteads 


ratio 


ratio 


.5 


.5 


.33333 


1.67 


1.86 


.5 


.66667 


.5 


1.26 


1.33 


•5 


.75 


.6 


1.15 


1.19 


.5 


.9 


.3)318 


1.02 


1.035 


.5 


.95 


. 90473 


1.005 


1.01 


,5 


.99 


.98020 


1.000 


1.000 


.?5 


.75 


. 42730 


1.21 


1.32 


•5 


.75 


.6 


1.15 


1.19 


■ 75 


.75 


.69231 


1.11 


1.13 



Table *U5 Improvement in chance of winning due to look-aheads 
used in ordinal strategies over the class of frames 
* with constant split ratio, 
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CHAPTER 5 
DEPRECIATION FACTORS AND NON-ORDINAL STRATEGIES 

Go far in this report, strategies have been evaluated 
os to their effect on the whole f^ame. In this chapter vie 
look at them in tneir effect from any riven position, and 
try to do this without detailed projections ol what will 
hannen later on, thufl oerhaps enablinr* us to decide which 
looK-afte.id ctrateriy t* use, move by mo" ; e# 

To this end we introduce the concent cf "denreci&tinn 
factors" to old in our evaluation of the relative contribution 
of "caution* and 'aggressiveness". For thn time beln;-, we 
will be concerned vxith the class of TGI names* with constant 
snlit ratio A, and in addition will concentrate at first on 
the cne look-ahead strategies for simplicity. 

Suppose we try to evaluate the orobability of winning 
under LSTRAT from a position which is a split point of value 
X, about which the only other information available is the 
value of its branches, which here will be assumed to be deter- 
mined by the constant split ratio A. The probability of 
winning is clearly less than X, because we can under the 
given strategy look only at a limited portion of the ,^arae tree 
below the point, and only move down a single path. This has 
already been seen to be true for all the limited look-ahead 
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strategies so lor considered for the cose where X 1 
and we are at the bw.inninfl of the .^o-Tie. However, sunnose 
we do "take X to be the probability of winning in 'such a 
case. We mir.ht impro-e our estimate by adding the actual, 
chance of winnin-* under LSTRAT on the first move from the 
Ijiven position, KB3 + (l-EBX)EAX, to the chance of movinr 
to a split point, (l-EBX)S, times the "Blue of that point, 
i-X, which is boinr; used as the estimate or the chance of 
winning from tnis second solit point. The resulting formula, 
call it V L , is 

V, - EBX + (l-EBX)EAX ■»• ( 1-EBX ) S ( .' JC ) 

= EBX * (1-EBX) (AX) - EBX + AX - EABX' 
Evaluating HSTRAT in the same way, we set 



V u - SAX + SAX + E(1-AX)(EBX+SBX} 

= AX + E(1-AX)BX = EBX + AX + EABX 3 

Our over-estimation of the chance of winning from a 
split point has ftiven rise to the interesting result that 

V = V . However, rather than puzzle over the exact reason 
L H 
it worKs out this way, let us try to correct that over- 
estimation. It would be very convenient if the actual chance 
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or winning fr*ra a split point was a zonstsnt, depending only 
on the strate/^y beinp used, times the value of the uoint. 
Such a constant could be called a depreciation factor"'. 
However, it is really too much to assume (in TSI) that cuch 
a constant exists, independent of the value of the position. 
>f hat is actually needed i3 probably a '"depreciation 
rune t ion" of E, Aj and X, Howeveri we can Ret an estimate 
of this function by assuming that it i£ constant, at least 
constant iron one ;r*ove to the next. Then letting K stsnd 
for the constant, XX would be equal to tne estimate of 
probability of winning which Is calculated analogously to the 
calculations of V T and V™ 

KX = EBX + (1-EBX)[EAX + SKAX) 

JJolvini', for K: 



[A - E^j 



.2 



v KBX + EXA - E'ABX'- _ E( l- EABX) 

L= X * ZSABX* - SAX T^SA-T^aBX 



As X approaches 0, K* (actually a function of E, A, 
and X) increases to a limit of E/{l-SA) 
For HSTRAT, we would have 

KX = EAX + SKAX + E(1-AX)[EBX + SKBX] 

K _ EAX + E g BX - E g ABX g E(A+EB j - E g A BX 

^ ~ X - SAX - ESBX + ESABX* T^dftKTFTOTx" 
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As X approaches O, K ;i increases to a limit of x" -S ?A»^ B 

Znterestinrly enough, the limiting values for K^ 

n 

and K L are » H and « L for TSD {Chapter 3). This 
surnrise has a good explanation, however. As X approaches 
0» the independent r.ane T3I **et closer to the dependent 
TOD (the (1-Xj 2'ictors you divide by when you discover 
losing endpoints, approach 1.) And in TSD 7J. and W„ are 
the depreciation factors, which in fact are constant throu^- 
out the 'namn, as can be seen if one r;ces o "or the analysis in 
chapter >. Because o: this constancy, they Rive the actual 
probability of winning when multiplied by the va 1 ue of the 
position. The top position has vaiue 1, so the chance of 
winning the r* A aine is simnly the depreciation factor itsolf. 

Now, if for Riven E and A we superscript the K's 
to let \C ~ limit K? , we have, from Theorem 3.3 

E • . _ E(A+_EB) 

T^SA K L - K H " 1 - S(A+EBJ 

It can be concluded from this that 

vX E - E g ABX EfA+EB) - E g ABX '--X 

*L ~ 1 - SA + ESABX - 1 - S(A+EB) 5 ESABX " *H 

because exactly the same terms are subtracted from each 
numerator, and exactly the same terms are added to each 
denominator. ■ 
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Now, since in T3I K* Is not constant for all 
position values X, X K? does not r,ive a precise estimate 
of the probability of winning from position X under 
L5TRAT, in fact tne real probability will be somewnere 
between X K* and X K* , It is intuitively olausible from 
the above ineaual Ities however, that the c franco of winning 
irom A under LSTRAT will be greater thin that under 
H?rPRAT, no natter what value X. ThJ^ is in nupport of the 
Claim A4Ji« in tiw last chapter that the tact that under 
HMTRAT there is a r.rc^itcr chance or winning on the £* rt 
move than under 1.3TRAT is only Indicative of the cumulative 
effect, and not of any real superiority or HSTRAT at that 

point in the same. 

We can further explain the kind of results summarized 
last chapter in table M>p as follows. Suppose you are at a 
position of value X small enough 30 that the limiting 
formulas K? and K„ approximately hold. Then the probabi- 
lities o' winning are, respectively 

XE md _XE(MEBi 



V - SA ma 1 - S(A*EBJ 

* 

Under ISTRAT the chance of winning on the next move, if X 
is small enough, is approximately EAX + EBX = EX, so the 
formula can be interpreted as representing the sum of the 
chances of winning on each successive move, the first term 
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beiap; XK and the ratio between succeeding terms beinp, 3A. 
(This was the way we *»ot the formula in the cose or TSD In 
chanter ^.Similarly, for HoTRAT the first term Is XK(A+BB) 
wd the ratio !>(A+KB). The ratio for H5TRAT is greater 
than that for LSTRAT, but this is outweighed by the Drepon- 
derence of the first term in the LSTHAT series. 

If this analysis is correct, the rntJo D(I)'D(I+1) 
described in chapter 4 should approach the ratios described 
in the 1 ar.t paragraph. This la shown In tabic ^,1 below 



HGTHAT: ^T^o} S<A+EB) U: 



TRAT: 




SA 



.5 


-5 


.37160 


.37500 


.?eyB2 


.25000 


.5 


.66667 


.11615 


.11667 


.33165 


.33333 


.5 


-75 


.13573 


.13750 


.37118 


-37500 


.5 


.9 


. 16771 


. *750O 


.11118 


.15000 


.75 


.5 


. 21867 


.21875 


. 12186 


. 12500 


.75 


.75 


.23311 


.23137 


. 18161 


.18750 



T%ble 5*1: Comparison between actual ratios between chonces 

of Winning on 9 th and 10 th moves with theoretical 
limits for -HSTRAT and L3TRAT in games of TSI 
with constant split ratio. 

* 

With the introduction of depreciation factors, it 

becomes possible to discuss intelligently non-ordinal strategies 
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which make use of the actual values of the branches of ft fork, 

not .Just their relative ranking. To do this we first dron 

the assumption thnt the split ratio is constant, and then add th« 

the 
assumption that depreciation functions .fust derived for 

constant split ratio A also hold approximately for a 

iictrjbutlon of snlit ratios merely with expected value A. 

The advantage of non-ordinal strategies, if there is 
any, will come from the possibility that, over the class of 
rano trees, the possibility of winnin;* from a split ooint 
Whose branches* valuer, are known may 'ary i little depending 
on how the top :a]ue tfl split between its successors. Thus, 
^iven two positions which are split points and whose split 
ratios are known, it may be possible th*t even though 
position X has value higher than position Y» the chance of 
winning from position X is lower, because its value splits 
in an unfavorable way. 

The ordinal strategies to which such considerations 

mifcht apply are HSTRAT, 2HSTRAT, and BSTRAT, for in all 

these we ordinarily can move to a split point whose split 

- 
ratio we have already seen in look-ahead. The use of this 

information to modify the strategy in the cases of BSTRAT 

and HSTRAT will be shown. The part of the strategies which 

tells where to look will remain the sajne, but the rules about 

where to move, rciven the results of the look-ahead, will be 
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altered. There is also the possibility of strategies with 
morn than one look-ahead, where the Miulta of the first 
look-ahends (the exact values of the branches seen) nay be 
used to determine where to look next. Such a strategy is 
the unlimited look-ahead strategy discucse'l in chapter 2, 
where that position with the highest 'alue of *11 those you 
have not yet seen, but Know the value of. Is looked at next* 
However i this is the onJy example of such non-crdinal 
strategies studied so far. 

One unfortunate aspect of the non-ordinal strategies 
vie do study is that they do not lend tnemselves to simple 
evaluation techninues* This is because an expected value of 
the split ratio, like the A vie used for ordinal strategies, 
is useless when we want to utilize the fact that the split 
ratio varies according to a probability distribution. 
Evaluation will have to be done by computer using the Monte 
Carlo methods whose reliability was brought into question in 
the last chapter. However if we maXe consistent use of the 
random number generator, we should be able to i?et an estimate 
of the amount of improvement non-ordinal information allows us. 
(and also its cost in computer time). 

So, to define these non-ordinal modifications; Under 
BSTRAT the depreciation factor is found as before 
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KX - <l\X t (l-KAX)KBX 4 E(l-AXj£KBX + (1-KBXJSKAX 



K X BX-E^ABX^ B-E^AB X 



Limit K* = K* « E/(l-:ifA+BEj), which combines tne best of 

both HSTRAT ejrl LoTRAT — the hi^h chance 0( winning on 
tne novej and the hitTh ratio between such chances on succeed- 
XtiF. novQB* But tni3 was to be expected, el le*'i3t in the 
limit. 

by usim* the depreciation "factor, we can ret an 
Idea of the probability of winning from thn ton of a fork 
w'tose SDlit ratio is Known. Define 



2 



k b (e,a,x) - i . i(AfB&) i SBSABX 



as a depreciation "function" giving the depreciated value of 
% position with value X. Then let our known fork have top 
value X and split ratio R. The chance of winning will 

be approximated by 

C B (E,A,X,R) - ERX + (1-ERX)E(1-R)X + E(1-RX)S KgfE^A, (l-R)X) 

+ (1-E(1-R)X)S KgtE^A.RX) 

{Note that we are st? 11 usin^ A as the expected value of the 
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sDlit ratios we don't know about yet*) BSTRAT would be 
modified only in the case when both hi*;h and low branches 
of your position have been found to be split points. Denote 
the value of the low branch by L, nnd tne hip,h branch ty H, 
and suppose that their split ratios are FL. and R. . We 
then compare C„(E,A,H,R H ) and C B (E,A,L,Hj) and move to 
the branch with the highest value for this. In all orobnbility 
this would be the H branch anyway, but ssjnole computer 
calculations show that if H/(H+L) < *55 there is at least a 
nor.slbill ty, correspondingly greater as the ratio annroaches 
, 5^» that tne I. branch may be chosen, 

Monte Carlo methods, implemented as described in 
chapter 1, yielded the following evaluation of BSTRAT and 
its modified version, as to chance of winning. In 10,000 
games, BSTRAT won .72^35 of the time, with a variation 
over repeated trials of only ,00010, The modified version 
B'STRAT won ,73200, with a variation of ,00030 over 
repeated trials. This is an improvement of approximately 
.00765. This is a considerable amount when you consider that, 
in Table J*. 3 for E » .5* A = .75* W 2L - W B is only ,00235* 
and so this modification probably makes BSTRAT the most 
effective two look-ahead strategy we have. This is bom out 
by a Monte Carlo evaluation of 2LSTRAT which yielded an 
estimate of W gL at .72810 + .00020, clearly less than the 

B 1 
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estimate for W R1 : . *: \ tf _ 






However B'STRAT, though more effective, is more 
costly than BSTRAT in the sense that it took slightly more 
than twice as much time to play 10,000 frames, due to the 
extra calculations involved in B'STRAT. This should 
probably not be taJren to seriously, since in most real /^ames 
the tim^ involved in such calculations would be much more 
seriously ftv^r-wei^hed by the time needed Lo simply generate 
the position 'alues, and nnyway, we could have skipped them 
exc^ot when H/fi^Hj ** .&5j which is the only Lime tney could 
be useful . 

Note that this non ordinal stratei'y is quite sophis- 
ticated in comparison to what mi(*ht have been suggested 
without the aid of depreciation factors. For instance* 

confronted with a situation such as in Figure 5.1* one mipht 
think to i? t o to the lower branch because its hiRh branch has 
the highest value of all points at its level. 




Figure 5.1: Example of part of n flame tree for T3I 
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However, since .6 (.4+. 6) > t $5 9 this would probably be 
wronr; assuming that our previous calculations are indeed 
annlicnble, and apnlyinr* such % simple rule over the loiv* 
run would surely be contra-productive. 

Turning to HoTRAT, a^ain we must nass up simple non- 
ordinai modi flcat ions, for instance: after looklnr; ahead to 
the .6 point, deciding to »*o to the .4 because .« is 
greater than . j. Instead, uninp the dcorfccintion factor 
derived abev;, w. ieJ'ine 

....... X[ 5(f-.+EBi - E^ABX] 

ft H (a,A»Kj ■ y etr-fl+t^-T-EST^/ 

ond, with R and T as alone 

C H (E,A,X f R) = EKR + SK H (K,A,XR) + E(l-XR) iE^l-Ri+SK^E, A,X{1-H) ) ] 

Supnose we are at a fork whose bottom joints are again H and 

L, and we find by look-ahead that the H branch splits, with 

solit ratio R. Then we can compare C H (t:,A,H,R) to 

EL + SK H (E>A,L) and move to the H branch only if the first 

va ue is the greater. This modification was evaluated by the 

Monte Carlo methods already described, with SK R (E,A,L) 

replaced by SC H (E,A,L ( Aj • which should be Improvement, although 

no proof has been found. The resulting fraction of wins was 

.67265 compared to .67070 as was obtained before for HSTRAT 

* 
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by this method, an increase of about ,00; ; 00, and nowhere 
near enough tc mnke it better than LSTRAT P which has an 
Advantage of nbout . 02&50 over HSTRAT. The cost was ar^aln 
a doubling of comouter time for 10,000 ^wies, but, likewise, 
a"ain wou*d have been reduced if in the cases where H (L^H) 
wan too rre-ot, the calculations of C had been omitted. 
Under t'lis strategy the cut-off noint would be about ,5^ 
for lan f & (in absolute terms) values of ll< decreasing to 
perhaps *^6 for H < .006, These cut off points were 
aetermined by computer evaluations o: C !or /nrious H and 
L, and why It should be what it is, and why it should deocnd 
on the value of H itself, are questions ooen to further 
research. 

The results of this research into non-ordinal strategies 
is summarized in Table 5.2 below 



««*.„, Fr^cUcn or wins l *B«S a jL3- 



Difference 



, j 



BETRAT 
HSTRAT 



.78*35 

. 67070 



. 7*i?O0 
. 67265 



.00765 
.00195 



Table 5.1: Comparison of Monte-Carlo Results ''or Ordinal 
and Non-Ordinal Strategies in Games of TSI 
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CHAPTER 6 
VARIANTS ON THEE SOLITAIRE 
AS A SUGGESTION FOR FURTHER STUDY 

Tne obvlOUS first sujyiestion for extending the results 
so f/ir reported would be to study in more detail the ordinal 
strategies in Tree EoJitaire with mor<* than two look-aheads, 
or perhaps strate/len which are not limited as to number ot 
looh-aheads but as to the number of ply deep in the rame tree 
they may look. 

Goin/r beyond the specific limitation to the rules of 
Tree Solitaire, the obvious second sujr^estion for extending 
the results would be to stop the restriction to binary name 
trees, and al'Jowinp, for the possibility of more than two moves 
from a ^iven position, Me could consider the case of ternary, 
tt-ary, etc. trees, or we could even let the number of branches 
at a given position vary accordinc to some probability 
distribution. A kind of tree solitaire could be defined on 
such trees, in our strategy definitions "low" could be 
replaced by second highest", and evaluation could procede in 
perhaps much the same way. 

However, rather than leap to such attempts in this 
last chapter, we shall continue our limitation to the binary 
trees for which our terminology has been developed, and 
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consider some variants Of tree solitaire preserving this 
characteristic. 

The principle failinr; of Tree Solitaire as a model 
for the kind of flames involving; look-ahead which we usually 
encounter, apoears to be the tact that the values of the 
nositions naturally decrease as you ^et further on in the 
,'*ame. This is the main reason we had to have depreciation* 
factors and the probable explanation of why a^r^ssive 
i/jiKftT did better tnon cautions HSTH/tT. Hov; can we ccrrect 
tnJn falling? 

There arc two >inds of modification, which can be 
effected independently. One is to change the r.ame tree and 
its generation, the other to change the "valuation" of the 
tree, the method of assifyiinp, values to its points. We would 
liKe to keep the basic appropxh of Reneratin^ the tree first, 
independently or the values which may later be assigned to 
it, a^thou^h these values may themselves reflect tree structure 
as is often the case in real *;ames and our evaluations of 
positions in them. 

An example of modifying the tree would be to have E» 
the probability that a point is an endpoint, be a function of 
its level in the tree (how many moves deep in the r*ame it is). 
This would probably not complicate matters much, and indeed, 
the diagram-computer program methods of chanter ^ mi^ht still 
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bo useable. In addition the essential feature of each point 
beinc Judged as E or an 5 independently of all the other 
noints is retained. However, by itseJf. this modification 
does not solve the problem referred to above, and this 
chapter will now Turther restrict itself to only those 
modifications which a?ter the valuation of the r;ame tree* 
leaving this suj^estion as ;*ust that -- a suggestion. 

Three nossib 1 e modified F t fm&z wil 1 be discussed, in 
ascend. iy order or usefulness. 

1, Giver, come probability distribution of numbers 
bctweon d '*nd \ t nnslrn valuer to the ondpolnts of the 
(finite) tree. The v.-xlue here reoresents the nrobability 
that the endpoint is a win. For every so? it point of the 
tree, let its value be calculated from the values o : the set 
of endooints which are its successors, immediately or farther 
down in the tree, end let it be the probability that at le^ast 
one of these is a win, i.e., the chance that there is a win 
beneath that split noint in the tree. If the set of successor 
endpoints has 3 numbers, with values A, B, and C, then the 
value of the paint would be A + B + C-AB-AC-BC + ABC. 

This does not solve the problem raised above, for the 
values of points decrease as you fto down the tree (although 
not indefinitely, as they can in TS), but it has one interest- 
ing facet. Consider the problem raised in chapter 2 of finding 
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thn least costly unlimited look-ahead strategy which ruajv^ntecs 
a win (if there are any wins tn the .vine)* The answer there 
for Tree Solitaire was to look first at the point with the 
highest value. In Came 1, if the distribution of endnoint 
values is sueh that the values are all equal to some constant 
A. the least costly strategy* in terrn^ of number o( positions 
you need to look at, is to look at the lowest valued point 
available. 

Proof: From the 7©3ua of a enlit point, we con, knowing A* 
calculate th<: number o; endpoints beneath It. The lower the 
value the fewer the endooints. However, the lower the number 
of endpoints beneath a point in the tree, the hirrher the 
proportion of such endpoints to the total number of points 
beneath that point. (If K is the number Of endpoints, there 
are K-l split points, counting the top one- K/3K-1 decrease: 
as K increases). Since all endpoints have equal probability 
of bein^ a win, tne least costly look-ahead procedure will be 
the one which wastes the least time in setting to look at 
endpoints. This is accomplished by looking at the lowest 
point, which can be expected, by the above consideration, to 
be the closest one to an endpoint. Q.E.D. 

The reason this works is that the value of a point is 
not independent of the structure underneath it in the tree, is 
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in laet a function of the number o' endooints* Even If the 
values of the endpoints were not equal but were assigned 
■ according to -a probability distribution as explained above, 
there would still be a denendence, from which some information 
About the tree structure could be obtained. Gu^h an arrange- 
ment i3 probably not at all unreasonable since such informa- 
tion is probably available in the valuations used in actual 
^ames, however, it is an exeircple of the non st^uidard coding 
onenomena rind complicates matters unduly (and arbitrarily /in 
n theoretic treatment even though it is valuable in nivin*' 
us an idea of how our expectations may ^o wronr in such 
s'tuations. H^re the best limited looh-ah^ad strategy would 
probably have to make use of sveh information, and. the 
evaluation of any strategy would have to take into account 
the dependency* An ability to count binary trees with a Riven 
number of endpoints might be needed, and there is no easy 
closed formula for that, 

2. Similar problems arise in this second grate, due 
ap;ain to the fact that values are assigned from the bottom of 
the tree up rather than from the top down, and thus can reflect 
tree structure. 

Assign values to the endpoint3 as in jjame 1« However 
this time let each split point have value equal to the 
average of the values of its immediate successors. We now 
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have avoided the i>rob)em of having naturally decreasing 
position "alues. However, ar;ain problems in evaluating 
strategies will arise because the value or a position does 
tell us a little about the tree structure benenth it. 
Altncu^h the eKnecfJd 'aue or the value or a split point is 
tne sone as that of an endooint, its variance, because it is 
an average, will be smaller, as a function of the number of . 
end-points bone^th it. Therefore a point with a vaiue very 
far from the mean is llfcely to have only a few endpoints 
beneath it, or be one Itself. 

This ^arae nos been mentioned not so mucn for its own 
imnortance, but because it su^ftests an idea for a ramc where 
7* lues for nositions are indeoendent of tne tree structure 
beneath then, and yet are '' likely" to be the average of their 
successors. 

3. Decide on a probability distribution 3 with mean 6. 
To the top of the tree assign a number T between and 1, 
Given a fork in the tree whose top value C has already been 
assigned, generate independently two numbers A and 3 
according to the distribution. Then the values of the branches 
are C f A' and C + B {i.e., the values of the branches are 
chosen according to the probability distribution 3 shifted over 
so that C Is Its mean). If a point is an endpoint with 
value p, the probability or its beinp a win is P if 
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< P < 1, 1 if P > 1. and if P < 0. 

Tne valuation in this p.ame is an idealization of 
the way which comouter uosition evaluation functions miRht 
be thought to work with regard to successive positions, pnd 
in that sense this K.eme should be a much fcntter aoDroxina- 
tionn to the rca! '' ( ajii&3 i'ivm pleyinr, Dro;gmrafl aj*e written to 
deal with, with tnis hone, the rest of this chapter is 
devoted to nhowinr how the ideas pnd technlnuos of the 
earUer chftptors ean bo used as a Btnrt on the evaluation of 
strateries in flames or form 3. 

To construct a diagram that mirjvt be pnnrooriate, we 
note that although the expected value oi either branch o^ a 
fork is the value C of the top point, the expected -.'clue of 
tta higher branch is somewhat greater than C and of the 
lower branch, somewhat less. Let us suppose that * is 
symmetric about its mean; then the expected value of the 
branches would be C +■ A and C - A, where A is determined 
from J*. This yields a diagram as in Figure 6.1 based on the 
assumption that the expected position values are always realized. 
The values on the lines can be filled in according to the 
ptrate*;y you are considering, and the computer programs of 
chapter 4 could easily be modified to evaluate all the 
strategies that were evaluated for TS. 
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T>5A 



T-3A 



Figure 6.1: Expected oosition values and possible moves In 
Gojne i- 

However, the reasonableness or the estimates or probability 
of winning tnuc obtained would depend on hovj the actual 
distribution 3 scattered it3 values and what affect this 
rni^nt nave. As in the case of Tree Golitalre, it's hard to 
tell whether usinr the expected values of position values to 
construct a diagram leaves out any essential considerations, 
and here we cannot fall back on the restriction to constant 
split ratio* The effect could be checked by Monte Carlo 
Methods, 

The results of sample hand calculations for T = 1/8 ■ 
A = 1/6 are summarized in Table 6.1, wnere after move 3 the 
formulas were too complicated to evaluate. 
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u Probability of winning 

MOVP HSTRAT LSTRAT 



.Pit 

.101 



1 . 


-361 


? 


.342 


3 


.1*9 



. * — 



SUM .752 -7fc' J 

T^ble 6,1; Hon! calculations of chances of winninr under 

HSTRAT n nd LSTSAT over the class of F t amcz of 
rorrn 3. 

It is noDarent th*t even though ISTRAT is ahead at the ^nd 
of '. smv<4S p H;sT»tAT will predominate when the surn is tnken 
0701"! n^.y, 10 move3« by which time the f^ame should be over* 

However, a more exact estimate can be obtained by 
cnlcul at !n;r, the probability of losing for thnt sum turn3 out 
to converge quite rapidly. See Table 6.? 

Probability of Losing 



Move 



H3TSAT 


LSTRAT 


.0556 


.139 


.0191 


.035 


.0069 


.000 


.0316 


.174 


ir. 0350 


.174 


.915 


.896 



STRAT 



1 .0556 .139 .167 

2 .0191 .035 .04? 

3 .0069 .000 .000 



SUM .0816 .174 .209 

Sum + correction factor. 0850 .174 .174 

l-m*-%*£2** -915 .UK .79! 

Table 6.2: Hand calculation" oJ chances of losing under H5TRAT 
LSTRAT t and UTRAT over the clnzs of games of form 3 
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STRAT was thrown in for sood measure to show the improvement 
due to look-ahead over the best one can do without any. The 
third term for X3TRAT and STRAT ia__0 because with A 
and K *3 they nro, from the ird mo n on -jnder these strate- 
gies you can ko only to positions with /alue greater thou or 
equal to 1. The conclusion one dr.iws iron this table in that 
H3TRAT is the best one-look-ahead strategy. It is reasonable 
to think that this generalizes to all Allowable values of A 
and E. How tn&t values of noints no lon/er need decrease 
later on in the ^amc there is a premium on caution" and 
"aggression" becomes "recklessness . 

Although a proof of HSTRAT's general superiority is 
not yet known, it can be shown for the limiting case where 
A *» 0, i.e. all positions have the same value T. (This is 
the same as p.ame 2 with all endpoints assigned the same value,) 

THEOREM 6.1; Let tf„ represent the chance of winninn under 

n 

HSTRAT> and W L the chance under LSTRAT, over the class of all 

gnmefl of form 3» in which every point of the game tree hab the 

same value T, < T < 1. Then W H > W L for all K (the 

probability that e point is an endpoint) * E < 1. 
■ 
Proof: Under HSTRAT the chance of winning at any particular 

position is 
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ET + E(l-T)ET - ET(1 + E - FTj 



She chnnce of continuing on Jn the fame is 



3 + E(l-T>'i = :J(1 + K - ET) 



u BT(2_+K-ETJ KT (H-5-ETj T( H-K-CTj 

00 ' A H L-OT/l 1 S-ETj " l-fT-Ej(l+E-BF7 "T^TEFr" 



For LSTHAT the chan=e of winninr, at any position is 



ET *• (l-ET)ET 



wvl the chance of continuing in the f,ame is 



(1 - ETJ3 = 1 - E - ET +.E" 



oO, 



gl'(y-sT) T(g-ET ) 

L " E(lt-T-ET) (1+T-ET, 



W, - 



u . u „ T(UE-ST) _ T(g-STj 
W H L T-TE+E 1+T-BT 



This will be > if 



1+E-ET)(1+T-ET) - (2-ET}(T+E-ET) > 



This can be simplified to 

l+T-ET+E+ET-E ? T-ET-ET ? +E 2 T ? -?T-2E+2ET+ET 2 +i; ? T-E ? T ? 
= 1-T-E+TE- (1-E)(1-T) > O 
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(bnraunn or the kriundw on K wl T/. ^. K*D. 

Kor A r:roat'.r tnon nnd -'iznttmira* thu jraxnc works 
lii:e di*y:rnm '"i.5. It can be shown that both the chance oi 
winning on the move and the chance of continuing past It sre 
Inp roved, for b'jth H5TRAT and LSTHfiT, but it is not irnroe- 
d'/vtely o,DD*irent wnich rains the most, I*: ia at least nlausibl 
to believe th*t HSTRAT maintains its superiority. Ferh^ns 
aof»tffln v : :/^ "Jftpreclfcllon actors will be inventablc 
(although here they mi:;ht be '' nttnr e clation factors", and can 
be used to solve tnis DroMon* and then tc .to on to pronose 
non-ordln^l strategies. 

At any rate, the avenues for rurther study Arc wide 

open, and hold prospects of more and varied results. 
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APPENDIX A: Programs ior eva^uatin:* strategies in chanter •* 

CALLER: This is the cilUnr. program for the rest of the 
programs, on CT3S. 

OUTPUT: This MAD external function is used by all the 

following strategy functions to sum nnd print their 
their rcsult3: the values D(I) for I - 1 through 
10 reorosentinf the chances of winning on eech of 
the first 10 moves, and the ratio between succeeding 
terns. The surs of the D(I) is incensed by P 
correction factor arrived at by assuming that tne 
rntio between successive terns, from 0(9} on, is 
a constant, COKR *■ D{10)/{1. -D(-).'D(10) ) - D{10) 

In uic fallowing MAD external functions which ealeulote 

Dfl for the various strategies discusse-1 in chanter U v;hen 

nroaented with /alues for E and A, V(I] represents the 

Value of the i tn point from the left in tne corresnondin* 

dinr-ram, at the level under consideration. T(Ij represents 

the probability of getting to a position with such a value at 

that level. All the programs start and end with the same 

statements, which are 

EXTERNAL FUNCTION 
PROGRAM COMMOii V.T.D, A,B,S,E, 
DIMENSION V(10),D(10),T(10) 
INTEGER J, K.L, II, 12, 13, I 1 * 



FUNCTION RETURN 
END OF FUNCTION 

nnd so only the differing parts will be listed. For each 

strategy the diagram will be presented, comments if necessary, 

and then the program listing. 
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LSI RAT 




30-a»bO 



Ml 



EifTRY TO LSTRAT. 
- E*A 

=- E-E 

l.-V(S) 

V(1)*T(1)+V(?) 



V 
V 
T 





THROUGH HI, FOR J=2,1,J.G. 10 

V P. = V 1)-B 

VI = Vl 1 J*A 
T ? « T' 1 J"S 

T 1 » *i 1)*S"(1.-V(2)) 
D Ji -- T l)*V(l)+T(2}*V(2j 
PRINT COMMENT $ L3TRAT? 
EXECUTE OUTPUT. 



. . : 
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HSTRAT 




H2 



M3 



Ml 



V 

V 1 

V 2 
T 
T 1 

T a 

D 1 



5KTRY TO HSTRAT. 

0. 

A 

B 

0. 

1. 

E-(l.-A) 

E"(T(1)-V{1)+T(2)*V{2)) 
THROUGH Ml, FOR J = 2, 1,J.G.10 
V(J+1} = V(J)»B 

THROUGH M2, FOR L = J,-l,L.LE.O 
V(L) - V(L)*A 
T(J+1) « E*S*(1.-V(J))T(J) 

through m3, for l - j.-1.l.le.0 
t(l) = s-t(l)+e-s*(1.-v(l-1})-t(l-1 

d(j) = 0. 

THROUGH Ml, FOR L - l.l.L.G.J+1 
D(J) - D(J)+V(L)"T(L)*E 
PRINT COMMENT $ HSTRAT$ 
EXECUTE OUTPUT, 
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2LSTRAT 




A P B 



The Tines **rc not connected at the AE and 

emohosize the i'oct that In T3I, under this 

would not move from B to AB and then to 

move E to AB only When you have seen by previous look- 

ahcvl that AB is n win. Otherwise th« nro^r&n is nuite 

similar to that lor L^THAT 



points t*u 
strategy, you 
A ? B etc. You 



Ml 



ENTRY TO LSTRAT 


V 


1 


= E'A 


V 


2 


= E-B 


T 


1 


= l.-E"B*(l.-S*A) 


T 


2 


= 1. 


T 


3 


= 1. 


D 


1 

mi 


= T(1)»V(1)+T(2)-V(2) 
JUGH Ml, FOR J = 2,1,J.G.10 



VIZ) » V 1 *B 

V 1 = V 1 *A 

TI2) m T l' *S+T("i)*S 

Tl 3 - T 1' *S 

Til' = T 1 "S # (1.-V(?)-V(?)*S-a; 

DlJ = T 1 -V(1)+T(?)*V(2) 

PRINT COMMENT $ ?LSTRAT$ 

EXECUTE OUTPUT. 
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2HS7RAT 



EO-XUS«i-flO 




IQrAK*StCt-jfgj] 

3 



'The diagram does not WOPfc like the ones before. Suppose y6U 
ore at AB and it is an s. If you came from B then you 
know that A 2 B is a split (else at B you would have moved 
to B ? J and so the chance of moving from AB to AB ? , Riven 
that you came from B is S^E(l-A'B). However, if you cane 
from A, this chance is E(1-A 2 B) + SEtl-A^B). Therefore 
you cannot, associate to each point in the diagram a single 
value renresentim" the chance of getting to a point with that 
value at that level, but need two numbers, one for each way 
of p.ettinr. there, and you must keep track of them separately. 
In the orosram R(I) is the possibility of coming from the 
upper left, T(I) from the upper right. 

DIMENSION R(10) 
ENTRY TO HSTRAT 

0. 

0. 

E-(1.-A)+S»E*(1.-A*A) 

0. 

1. 

0. 



R U 

R 1 

R 3 

T 

T 1 

T 2. 



- 73 - 



, 



u-.- 



Ma 



Ml 



i_ & Vj 



V = 0. 

VI' -A 

Vi'2 - B 

0(1) = E*(V(1)+R(2)*V(2)) 

THROUGH Ml, FOR J i P.1.J.G.10 

v(j-n) - v(j)*a 

THROUGH 142, FOR L - J, -l.L. LE. 0. 



* i . 



3*1) - R(J)*E*(l.-V(J))-rS»(l.-V{j)*A)>*S 
J+l/ = 0. 
THROUGH M3. FOR L - ,-l,L. LE. 9 
TfL -. S-(T(LJ+R(L)) 
R(L ■ T(L-1)*S*3*E* , (1.-V{L-1)*A) 
1 ;-R T-J) # E*S»((1.-V(L-1 ))+o'(l.-V(L-lj'A^ J 
J(J) - O. 

THHOUGII Ml, FOR L = 1, 1,1,, G- J+l . 
n{J) = li(J / +V(L)-E*(T{L)i-H(L)i 
PKINT COMMENT $ 2HSTRATJ 
EXECUTE OUTPUT. 
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-"PSA I 

® / A® 







ooHSTpflT 



ooLSTRAT 



Ml 



M2 



ENTRY TO LIMITS. 
G - l./(l.-3*A) 
D(1J - E*A*G 
Tjl) - l.-D(l) 

V{1> - E 

THROUGH HI, FOR J - 2,1,J.G.10 

V(l) - V(1)*B 

C = V(1)*G*A 

D(J) - T(1)»(C*S+V(1)) 

Tfl) = (l.-C)«S-T(l) 

PRINT COMMENT A HLIMIT* 

EXECUTE OUTPUT. 

1(1) = 1. 

V(l) = E 

THROUGH M2, FOR J « 1,1,J.G. 10 

C - V(1)»B*G 



D(.J) - 
V(li - 
Tfl 



T< 



(C+(1.-C)*V(1)*A) 



A 



-C)«S 



PRINT COMMENT $ LLIMIT$ 
EXECUTF. OUTPUT, 



- 75 - 



BSTRPT 



SO-EB> 




t - 



Ir.iz disarm) Ic oned a bit differently fyon the rest. At each 
noint «he enanca of winning, not at tnat noint, but at its two 
successors if troy exist, is computed. If the -'alue of the 
Doint is X, the formula for this is EXB h (l-EXB)SXA, Thus 
the numbers on the lines indicate the orobabillty or setting 
to the points and finding that they solit. This modification 
is necessary because of the way BSTRAT works, and is related 
to the modifications for 2HSTRAT & 2LSTRAT. 



1 


^HTRY TO BSTRAT 


s 


D 


1 


= E»B*(l-E*B}*E*A 


r 


T 





= 0. 


- 


T 


1 


= S«(1.-E"B) 
- S"E»(1.-A) 




T 


? 




V 


= 0. 




VI 


= A 




V(P 


= B 




THROUGH Ml, FOR J ■ ■ 2,1, J. 0.10 




D(JJ = 0. 




Tl 


IRC 


-UGH M2, FOR L -- l,l f L.G.J 



M? 



M3 



Ml 
Ml 



D(J) - D(J)+E-V(L)'T(L)*(B+(1.-EB*/(L))'A) 

WHENEVER J.E. 10, TRANSFER TO Ml 

V(J+1J - V(J)-B 

THROUGH 143, FOR L = J.-1.L.LE.0 

V(L) = V(L)-A 

T(J+1) = T(J)-E-S»(1.-V(J)) 

THROUGH M-, FOR L = J,-1,L. LE.O 

T(L) = S*((1.-E'V(L))"T(L)+T(L-1) ,, E*(1.-V(L-1)) 

CONTINUE 

PRINT COMMENT $ BSTRAT$ 

EXECUTE OUTPUT. 
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APPENDIX B: Sample Monte Carlo Evaluation Program 

The following MAD program was used on a CTSS system 
to evaluate ~~ HSTRAT with non-ordinal modif icat'ons, over the 

class of TSD r.ames, by playing through a series of randomly 

■ 

.-renerato-l frames ysinp that strater,y. 

IMTEGfcl* I, J 
READ FORMAT F,J 
V2CT0RVALUES F=$IV$ 
THROUGH Ml, FOR 1-1,1, I. G.J 
Ml A-HAHMO. (XJ 

READ FORMAT Fl.K.NGAME 
VECTOR VALUES F1=*F5. 5>F5. 0'4 
G-O. 

T=0. 
TN-O. 
M3 WHENEVER G.GE.NGAME, TRANSFER TO END 
Z.l = l. 

Ml A«RANH0. (X)/2. + ,5 

T-T+A 

TH-TN+1 . 
MP X2-X1*A 

X3-X1-XP 

AiRANNO. (X) 

WHKHEVER A.G.E, TRANSFER TO M5 

A-TANHO. (X) 

WHENEVER A.G.X2, TRANSFER TO M6 
WIN W=H+1. 
DOSE G-0+1. 

TRANSFEP TO M3 
M5 A=RANNC. (X)/».+.5 

T=T+A 

TN-TN+1. 

H=E*X3+{1.-E)-C. (X3..75J-C. (XS.A) 

WHENEVER H.O.O. , TRANSFER TO M6 

X1=X2 

TRANSFER TO M2 
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M6 A=RANNO. (X) 

WHENEVER A.O.E, TRANSFER TO MIO 
A-RAWMO. (X) 

WHENEVER A.G.X3, TRANSFER TO LOSE 
TRANSFER TO WIN 

MIO X1=X3 

TRANSFER TO Ml 
END SCORE- W/G 

AVS-T/TH 

PRINT FORMAT F2, SCORE, AVE 

VECTOR VALUES F2-$2F8.5*$ 

EXECUTE EXIT. 

INTERNAL FUNCTION K. (Y)-Y*{E*( . 75+E*. 25)-E"*E*. l875*X 
1 {l.-5*(.75+E*.25)+E*S*.l875*Y} 

INTERNAL FUNCTION C. <U, R}-E-U*R+S*K. (U*R)+E»(l.-U»Rj 
1 »(S"U"(1.-R)+S"X. (U-(l.-R;)) 

SKD OF PROGRAM 



■ 



I 
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